Wormhole: Reliable Pub-Sub to Support Geo-replicated Internet Services
نویسندگان
چکیده
Wormhole is a publish-subscribe (pub-sub) system developed for use within Facebook’s geographically replicated datacenters. It is used to reliably replicate changes among several Facebook services including TAO, Graph Search and Memcache. This paper describes the design and implementation of Wormhole as well as the operational challenges of scaling the system to support the multiple data storage systems deployed at Facebook. Our production deployment of Wormhole transfers over 35 GBytes/sec in steady state (50 millions messages/sec or 5 trillion messages/day) across all deployments with bursts up to 200 GBytes/sec during failure recovery. We demonstrate that Wormhole publishes updates with low latency to subscribers that can fail or consume updates at varying rates, without compromising efficiency.
منابع مشابه
Local Building Blocks for a Scalable Pub/Sub Infrastructure
A publish-subscribe (pub/sub) infrastructure [7] supports the dissemination of varied information to a large population of users with individual preferences. In this paper we focus on topic-based pub/sub systems, where users can subscribe to topics of interest, and receive all updates related to these topics. One notable application of such systems is real-time dissemination of trading data to ...
متن کاملCOPSS-lite: Lightweight ICN Based Pub/Sub for IoT Environments
Information Centric Networking (ICN) is a new networking paradigm that treats content as the first class entity. It provides content to users without regards to the current location or source of the content. The publish/subscribe (pub/sub) systems have gained popularity in Internet. Pub/sub systems dismisses the need for users to request every content of their interest. Instead, the content is ...
متن کاملA Distributed Publish-Subscribe Architecture For XML-Based Event Dissemination
The aim of this study is to investigate how a pub-sub architecture can be efficiently empowered with XML-based filtering capability in a distributed environment. An XML based pub-sub system mainly focus on active, selective, asynchronous dissemination of timely, personalized and dynamic information represented and modelled by XML. It supports improved filtering flexibility and expressiveness as...
متن کاملAdapting Distributed Real-Time and Embedded Pub/Sub Middleware for Cloud Computing Environments
Enterprise distributed real-time and embedded (DRE) publish/subscribe (pub/sub) systems manage resources and data that are vital to users. Cloud computing—where computing resources are provisioned elastically and leased as a service—is an increasingly popular deployment paradigm. Enterprise DRE pub/sub systems can leverage cloud computing provisioning services to execute needed functionality wh...
متن کاملA Coordination and Discovery Service for QoS-enabled Data-Centric Publish/Subscribe in Wide Area Networks
A special class of Internet of Things called the Industrial Internet of Things (IIoT) operates in a large, distributed and dynamic environment comprising sensors all the way to large server clusters. A key requirement for IIoT is a scalable messaging service that supports multiple quality of service (QoS) properties, such as timeliness and resilience. Although existing pub/sub standards, such a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015